feat: make host mode directory paths configurable #7225

codefromthecrypt · 2025-10-13T17:56:49Z

What this PR does / why we need it:

Before, all paths were hard coded to start with /tmp/envoy-gateway, making parallel runs impractical, especially when certs change.

This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.

The implementation introduces four configurable directory paths:

configHome: certificates and configuration files (default: ~/.config/envoy-gateway)
dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway)
stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway)
runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID})

Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.

These paths are propagated to func-e which creates subdirectories as needed:

dataHome/envoy-versions/ for Envoy binaries
stateHome/envoy-runs/{runID}/ for per-run logs
runtimeDir/{runID}/ for per-run runtime files

codefromthecrypt · 2025-10-13T18:16:35Z

license check fails due to not indexed yet. I'll fix the other things later https://deps.dev/go/github.com%2Ftetratelabs%2Ffunc-e

codecov · 2025-10-14T10:57:53Z

Codecov Report

❌ Patch coverage is 63.10680% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.99%. Comparing base (a2ce000) to head (bf2fb0c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/infrastructure/host/infra.go	0.00%	15 Missing ⚠️
internal/cmd/certgen.go	35.29%	11 Missing ⚠️
internal/globalratelimit/runner/runner.go	66.66%	5 Missing and 1 partial ⚠️
internal/gatewayapi/runner/runner.go	85.71%	1 Missing and 1 partial ⚠️
internal/infrastructure/host/paths.go	91.66%	1 Missing and 1 partial ⚠️
internal/xds/runner/runner.go	81.81%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7225      +/-   ##
==========================================
+ Coverage   71.97%   71.99%   +0.01%     
==========================================
  Files         229      230       +1     
  Lines       33201    33280      +79     
==========================================
+ Hits        23897    23960      +63     
- Misses       7573     7579       +6     
- Partials     1731     1741      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codefromthecrypt · 2025-10-14T11:10:51Z

🤞 build is green now

codefromthecrypt · 2025-10-14T11:59:16Z

@arkodg @nacx @missBerg fyi I am off next week but wanted to eliminate this nagging tech debt for good. This PR uses XDG conventions (like prometheus, goose etc), which map into the same change made in func-e. The result is no more hard-coded /tmp/envoy-gateway and flexibility to change things in docker etc.

Most importantly, envoy-ai-gateway is blocked on this because the certs directory is hard-coded here. This causes problems as when you run tests that update certs it breaks your host configuration, as there's no way to overwrite it before this change.

So, TL;DR; this isn't just a random TODO fix, this is a big deal for sustainability forward and we should merge soon, especially as I can finish up the whole thing soup-to-nuts including ai gateway if folks can prioritize it this week!

codefromthecrypt · 2025-10-14T12:22:10Z

OIDC_with_BackendCluster is a very flaky test.

Issue flaky: FAIL TestE2E/OIDC http_route_with_oidc_authentication #3898: "flaky: FAIL TestE2E/OIDC http_route_with_oidc_authentication" was filed and later closed
PR workaroud for the flaky oidc e2e test #4603 (Nov 2024): A workaround was specifically added to handle flaky OIDC tests, which deletes and recreates the SecurityPolicy when failures occur
PR Use BackendCluster to represent OIDCProvider #4227 (Oct 2024): The OIDC_with_BackendCluster test was added as part of implementing BackendCluster support for OIDC providers

Please kick the build. Maybe we can prioritize deflaking this after merge, especially as you can use a clean certs directory after. The certs being hard-coded directory smashed by other tests, caused flakes in ai-gateway also.

codefromthecrypt · 2025-10-14T12:33:32Z

incidentally this flake reminded me certs should be in the ConfigHome as they are configuration bound (for exactly the same reason as flakes when one test needs certs for its config). changing this now

codefromthecrypt · 2025-10-14T12:41:44Z

updated wrt cert location

This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility. The implementation introduces four configurable directory paths: - configHome: certificates and configuration files (default: ~/.config/envoy-gateway) - dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway) - stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway) - runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID}) Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts. These paths are propagated to func-e which creates subdirectories as needed: - dataHome/envoy-versions/ for Envoy binaries - stateHome/envoy-runs/{runID}/ for per-run logs - runtimeDir/{runID}/ for per-run runtime files The changes include path resolution helpers, updated runners to use configurable paths, comprehensive test coverage, and updated documentation. Signed-off-by: Adrian Cole <[email protected]>

jukie · 2025-10-16T03:59:15Z

/retest

zirain · 2025-10-16T05:19:21Z

/retest

arkodg

LGTM thanks

zirain · 2025-10-24T01:44:33Z

@codefromthecrypt can you fix the conflict?

mathetake · 2025-10-24T03:16:10Z

adrian is ooo until next week i think

mathetake · 2025-10-24T03:16:48Z

if the conflicts are not huge then you maintainer folks can push directly to the branch then merge i guess

Signed-off-by: Arko Dasgupta <[email protected]>

#1368) **Description** This refactors aigw to use distinct directories following XDG Base Directory Specification conventions: - --config-home/$AIGW_CONFIG_HOME: Configuration files (default: ~/.config/aigw) - --data-home/$AIGW_DATA_HOME: Envoy binaries via func-e (default: ~/.local/share/aigw) - --state-home/$AIGW_STATE_HOME: Run logs and state (default: ~/.local/state/aigw) - --runtime-dir/$AIGW_RUNTIME_DIR: Ephemeral files like UDS (default: /tmp/aigw-${UID}) This separation aligns with XDG principles where configuration, data, state, and runtime files are independently configurable for different storage tiers. This is particularly useful for Docker deployments to map volumes appropriately. This also adds --run-id/$AIGW_RUN_ID to override the default YYYYMMDD_HHMMSS_UUU timestamp format with a custom identifier. Setting this to '0' enables predictable paths for Docker/Kubernetes single-run scenarios. **Related Issues/PRs (if applicable)** once envoyproxy/gateway#7225 is merged we have some maintenance to remove the /tmp/envoy-gateway/certs tech debt --------- Signed-off-by: Adrian Cole <[email protected]>

This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility. The implementation introduces four configurable directory paths: - configHome: certificates and configuration files (default: ~/.config/envoy-gateway) - dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway) - stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway) - runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID}) Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts. These paths are propagated to func-e which creates subdirectories as needed: - dataHome/envoy-versions/ for Envoy binaries - stateHome/envoy-runs/{runID}/ for per-run logs - runtimeDir/{runID}/ for per-run runtime files The changes include path resolution helpers, updated runners to use configurable paths, comprehensive test coverage, and updated documentation. Signed-off-by: Adrian Cole <[email protected]> Signed-off-by: Lin Moskovitch <[email protected]>

mathetake · 2025-10-27T23:17:39Z

internal/cmd/certgen.go

 	cmd.PersistentFlags().BoolVarP(&local, "local", "l", false,
 		"Generate all the certificates locally.")
+	cmd.PersistentFlags().StringVar(&dataHome, "data-home", "",
+		"Directory for certificates (defaults to ~/.local/share/envoy-gateway)")


shouldn't this be ~/.config/envoy-gateway given that CertDir is written like this? cc @codefromthecrypt

// CertDir returns the certificate directory path (under ConfigHome). func (p *Paths) CertDir(component string) string { return filepath.Join(p.ConfigHome, "certs", component) }

the other way works too (changing CertDir to use ~/.local/share/envoy-gateway)

codefromthecrypt requested a review from a team as a code owner October 13, 2025 17:56

codefromthecrypt force-pushed the feat/configurable-xdg-paths branch from 6383591 to 5398829 Compare October 14, 2025 10:49

codefromthecrypt closed this Oct 14, 2025

codefromthecrypt deleted the feat/configurable-xdg-paths branch October 14, 2025 10:49

codefromthecrypt restored the feat/configurable-xdg-paths branch October 14, 2025 11:10

codefromthecrypt reopened this Oct 14, 2025

codefromthecrypt force-pushed the feat/configurable-xdg-paths branch from 5398829 to 66871b3 Compare October 14, 2025 12:40

codefromthecrypt mentioned this pull request Oct 14, 2025

refactor: separate XDG directories for config, data, state and runtime envoyproxy/ai-gateway#1368

Merged

codefromthecrypt force-pushed the feat/configurable-xdg-paths branch from 84779d3 to 935470e Compare October 14, 2025 17:28

codefromthecrypt mentioned this pull request Oct 15, 2025

fix: data race in host infrastructure proxy context map #7176

Closed

jukie requested a review from a team October 16, 2025 04:00

arkodg previously approved these changes Oct 24, 2025

View reviewed changes

arkodg requested review from a team October 24, 2025 01:40

arkodg added this to the v1.6.0-rc.1 Release milestone Oct 24, 2025

zirain previously approved these changes Oct 24, 2025

View reviewed changes

Merge branch 'main' into feat/configurable-xdg-paths

bf2fb0c

Signed-off-by: Arko Dasgupta <[email protected]>

arkodg dismissed stale reviews from zirain and themself via bf2fb0c October 24, 2025 04:04

arkodg merged commit 20cac51 into envoyproxy:main Oct 24, 2025
32 of 33 checks passed

codefromthecrypt deleted the feat/configurable-xdg-paths branch October 27, 2025 04:56

mathetake reviewed Oct 27, 2025

View reviewed changes

mathetake mentioned this pull request Oct 27, 2025

fix: use consistent cert dir between certgen and standalone run #7351

Merged

feat: make host mode directory paths configurable #7225

feat: make host mode directory paths configurable #7225

Uh oh!

Conversation

codefromthecrypt commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codefromthecrypt commented Oct 13, 2025

Uh oh!

codecov bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codefromthecrypt commented Oct 14, 2025

Uh oh!

codefromthecrypt commented Oct 14, 2025

Uh oh!

codefromthecrypt commented Oct 14, 2025

Uh oh!

codefromthecrypt commented Oct 14, 2025

Uh oh!

codefromthecrypt commented Oct 14, 2025

Uh oh!

jukie commented Oct 16, 2025

Uh oh!

zirain commented Oct 16, 2025

Uh oh!

arkodg left a comment

Choose a reason for hiding this comment

Uh oh!

zirain commented Oct 24, 2025

Uh oh!

mathetake commented Oct 24, 2025

Uh oh!

mathetake commented Oct 24, 2025

Uh oh!

Uh oh!

mathetake Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

mathetake Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

mathetake Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codefromthecrypt commented Oct 13, 2025 •

edited

Loading

codecov bot commented Oct 14, 2025 •

edited

Loading